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anisotropic reflection model which has different reflection intensities respectively with 
respect to light source positions or viewpoint positions. 

These methods are described in "Hair expression using an anisotropic 
reflection model" by Yamana and Suenaga, Singakugihou, PRU87-3, pp. 115 to 120, 
5 1989. In these methods, expression of a hair style is just the shape of a polygon model 
itself, and the hair style can be changed by changing the shape of the polygon model. 

A proposal has been made for a method of expressing hair by a particle system. 
In the particle system, when a large quantity of particle objects are dealt with, 
characteristics such as weight and the like of each particle are given as parameters, and 
10 particles are generated under a certain rule. Thus, a large quantity of particle objects 
can be expressed easily. 

Meanwhile, the present applicant has proposed a method of approximating 
] every hair of a hairstyle to a curve, as shown in FIG. IB, in Japanese Patent 

^ Application No. 10-164545, for example. This method is successful in natural 

3 1 5 expression of a hair style image. 

A method of designing a hair style by editing together a plurality of hairs 
yi approximated to a curve is described in "Development of a hair design system using 

fy tuft models" by Kishi, Saegusa, and Morishima, in Shingakugihou, MVE97-102, pp. 

67-74, 1998. 

20 It is difficult for the method of approximating the entire hair style to a polygon 

model to express a complicated hair style. In this method, movements (flows) of hair 
cannot be expressed. 

On the other side, the method of expressing hair by a particle system and the 
method of approximating every hair to a geometric figure can express any arbitrary 

25 hair style and movements of hair. However, in the method of expressing hair by the 
particle system, parameters for particles expressing hairs must be controlled to control 
the hair style, so that intuitive control is difficult. It is therefore difficult to generate an 
intended hair style. 

In the method described in "Development of a hair design system using tuft 
30 models" by Kishi, Saegusa, and Morishima, in Shingakugihou, MVE97-102, pp. 67- 
74, 1998, an intended hair style can be generated relatively easily. This method, 
however, has a problem in that hair cannot be changed to intended hair styles 



corresponding to a plurality of different head shapes although hair corresponding only 
to one fixed head shape can be changed to an intended hair style. 

SUMMARY OF THE INVENTION 
One of the objects of the present invention is changing a hair style consisting of 
hairs of which is expressed by CG, so as to correspond to a number of different head 
shapes. 

The present invention provides an image processing apparatus for synthesizing 
a hair image with a three-dimensional shape image of a head part, to generate a hair- 
style-matched image, including: a storage part for storing a hair-style data piece; a read 
out part for reading out the hair-style data piece stored in the storage part; a mapping 
part for mapping a hair data piece corresponding to a hair and contained in the hair- 
style data piece read out by the read out part, at a predetermined position on the three- 
dimensional shape image of the head part; and a generation part for generating the 
hair-style-matched image, based on a mapping result obtained by the mapping part. 

As the hair data piece, it is possible to use three-dimensional curve data which 
is constructed by a number of control points. 

As the hair data piece, it is possible to use polygon data which is constructed 
by a number of polygons. 

The storage part stores the hair-style data piece constructed by hair data pieces 
arranged on a two-dimensional array. In particular, the storage part stores the hair- 
style data piece constructed by data pieces arranged on a two-dimensional array 
corresponding to a projected image obtained by projecting a three-dimensional hair 
style expressed by a columnar coordinate system, on a two-dimensional coordinate 
system. 

The apparatus according to the present invention further includes an 
interpolation part for interpolating the hair data piece at the predetermined position, 
with use of a hair data piece close to the predetermined position. The interpolation 
part interpolates the hair data piece, based on group information contained in the hair 
data piece close to the predetermined position. 

The mapping part includes a coordinate conversion part for converting the hair 
data piece expressed by a first coordinate system into data of a second coordinate 
system by which the three-dimensional shape image of the head part is expressed. 



The coordinate conversion part converts the hair data piece expressed by the 
first coordinate system into data of a local coordinate system having an origin on a 
surface of the head part, and thereafter converts the data into data of the second 
coordinate system by which the three-dimensional shape image of the head part is 
expressed. 

When the hair data piece expressed by the first coordinate system is converted 
into the data of the second coordinate system by which the three-dimensional shape 
image of the head part is expressed, the coordinate conversion part executes at least 
one of coordinate axis rotation and origin shift, based on a random number. 

Also, the present invention provides an image processing method for 
synthesizing a hair image with a three-dimensional shape image of a head part, to 
generate a hair-style-matched image, including: a storage step of storing a hair-style 
data piece; a read out step of reading out the hair-style data piece stored by processing 
in the storage step; a mapping step of mapping a hair data piece corresponding to a 
hair and contained in the hair-style data piece read out by processing in the read out 
step, at a predetermined position on the three-dimensional shape image of the head 
part; and a generation step of generating the hair-style-matched image, based on a 
mapping result obtained by processing in the mapping step. 

In the processing in the storage step according to the present invention, the 
hair-style data piece constructed by hair data pieces arranged on a two-dimensional 
array is stored. 

Also, in the processing in the storage step, the hair-style data piece constructed 
by data pieces arranged on a two-dimensional array corresponding to a projected 
image obtained by projecting a three-dimensional hair style expressed by a columnar 
coordinate system, on a two-dimensional coordinate system, is stored. 

The method according to the present invention further includes an interpolation 
step of interpolating the hair data piece at the predetermined position, with use of a 
hair data piece close to the predetermined position. 

In processing in the interpolation step, the hair data piece is interpolated, based 
on group information contained in the hair data piece close to the predetermined 
position. 



The processing in the mapping step includes a coordinate conversion step of 
converting the hair data piece expressed by a first coordinate system into data of a 
second coordinate system by which the three-dimensional shape image of the head part 
is expressed. 

5 In processing in the coordinate conversion step, the hair data piece expressed 

by the first coordinate system is converted into data of a local coordinate system 
having an origin on a surface of the head part, and thereafter, the data is converted into 
data of the second coordinate system by which the three-dimensional shape image of 
the head part is expressed. 
Q 10 When the hair data piece expressed by the first coordinate system is converted 

into the data of the second coordinate system by which the three-dimensional shape 

sss 

03 image of the head part is expressed, at least one of coordinate axis rotation and origin 

hj shift is executed, based on a random number, in processing in the coordinate 

y - conversion step. 

€3 15 Further, the present invention provides a recording medium which records a 

01 

Li computer-readable program for image processing of synthesizing a hair image with a 

Jj] three-dimensional shape image of a head part, to generate a hair-style-matched image, 

pj the program including: a storage step of storing a hair-style data piece; a read out step 

of reading out the hair-style data piece stored by processing in the storage step; a 
20 mapping step of mapping a hair data piece corresponding to a hair and contained in the 
hair-style data piece read out by processing in the read out step, at a predetermined 
position on the three-dimensional shape image of the head part; and a generation step 
of generating the hair-style-matched image, based on a mapping result obtained by 
processing in the mapping step. 
25 In the processing in the storage step, the hair-style data piece constructed by 

the hair data pieces arranged on a two-dimensional array is stored. 

In the processing in the storage step, the hair-style data piece constructed by 
data pieces arranged on a two-dimensional array corresponding to a projected image 
obtained by projecting a three-dimensional hair style expressed by a columnar 
30 coordinate system, on a two-dimensional coordinate system, is stored. 
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According to the present invention, the program further includes an 
interpolation step of interpolating the hair data piece at the predetermined position, 
with use of a hair data piece close to the predetermined position. 

In processing in the interpolation step, the hair data piece is interpolated, based 
5 on group information contained in the hair data piece close to the predetermined 
position. 

The processing in the mapping step includes a coordinate conversion step of 
converting the hair data piece expressed by a first coordinate system into data of a 
second coordinate system by which the three-dimensional shape image of the head part 
p 10 is expressed. 

j~: In processing in the coordinate conversion step, the hair data piece expressed 

03 by the first coordinate system is converted into data of a local coordinate system 

having an origin on a surface of the head part, and thereafter, the data is converted into 
data of the second coordinate system by which the three-dimensional shape image of 
C3 1 5 the head part is expressed. 

When the hair data piece expressed by the first coordinate system is converted 
into the data of the second coordinate system by which the three-dimensional shape 
image of the head part is expressed, at least one of coordinate axis rotation and origin 
shift is executed, based on a random number, in processing in the coordinate 
20 conversion step. 

In the above-described image processing apparatus and method according to 
the present invention, hair-style data is stored, and the stored hair-style data is read out. 
Hair data corresponding to a hair and included in the read out hair-style data is mapped 
at a predetermined position on a three-dimensional shape image of a head part. Based 
25 on the mapping results, a hair-style-matched image is generated. The stored hair-style 
data is constructed by pieces of hair data arranged on a two-dimensional array. 

Additional features and advantages of the present invention are described in, 
and will be apparent from, the following Detailed Description of the Invention and the 
Figures. 

30 BRIEF DESCRIPTION OF THE FIGURES 

FIGS. 1A and IB are views showing an example of conventional hair 
expression. 
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FIG. 2 is a block diagram showing a structural example of a hair synthesis 
system to which the present invention is applied. 

FIG. 3 is a flowchart explaining operation of the hair synthesis system. 

FIG. 4 is a view for explaining the operation of the hair synthesis system. 
5 FIG. 5 is a block diagram showing a structural example of processing in the 

hair- style mapping processing section shown in FIG. 2. 

FIG. 6 is a flowchart for explaining processing in the hair-style mapping 
processing section. 

Li FIGS. 7 A and 7B are views for explaining hair- style data in which hair data is 

S3 10 two-dimensionally arrayed. 

U FIG. 8 is a view for explaining mapping position calculation in the step S13 

shown in FIG. 6. 

yJ FIG. 9 is a view for explaining hair data interpolation processing in the step 

yj 

5 S 1 5 shown in FIG. 6. 

JBSS. 

Jji 15 FIGS. 10A and 10B are views for explaining coordinate conversion in the step 

y i 

M S16 shown in FIG. 6. 

U1 

q FIG. 1 1 is a view showing a data structure of the hair-style database shown in 

nj FIG. 1. 

FIG. 12 is a view for explaining interpolation processing using group 
20 information included in hair data. 

FIGS. 13A and 13B are views for explaining coordinate axis rotation in 
coordinate conversion. 

FIGS. 14A and 14B are views for explaining coordinate axis shift in coordinate 
conversion. 

25 FIGS. 15A and 15B are views for explaining of mapping to the three- 

dimensional model of a head part with an arbitrary changed. 

FIG. 16 is a view showing a columnar coordinate system when hair-style data 
is generated. 

FIG. 17 is a view showing a structural example of a personal computer which 
30 realizes an image processing apparatus. 
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DETAILED DESCRIPTION OF THE INVENTION 
A hair synthesis system to which the present invention is applied will be 
explained with reference to FIG* 2. This hair synthesis system includes: an image 
pickup device 1 which picks up an image of a neck part (the part above the neck, 
including the neck) of a person; an image processing device 2 for generating a three- 
dimensional shape model of the neck part, based on the image of the picked-up neck 
part, synthesizing a CG of an arbitrary hair style, and generating a hair-style-matched 
image; and a display device 9 for displaying the generated hair-style-matched image. 
If this hair synthesis system is installed in a beauty salon, a barber's shop, or the like, it 
can be used to assist customers to select hair styles or so. 

The image pickup device 1 of the hair synthesis system combines a three- 
dimensional shape measurement device such as a stereo camera, a range finder, or the 
like, and a two-dimensional image input device such as a CCD (Charge Coupled 
Device) or the like. In case of a stereo camera, the three-dimensional measurement 
device of the image pickup device 1 picks up an image of the head part of a person (a 
customer of a beauty salon or the like) as a target object, several times from a different 
kinds of direction, and performs predetermined image processing on the image, to 
obtain a three-dimensional shape of the face part (which is a part obtained by 
removing a hair part from the neck part). In case of a range finder, the neck part of the 
person as the target object is subjected to laser scanning, thereby measuring the 
distance, to obtain a three-dimensional shape of the neck part. The two-dimensional 
input device of the image pickup device 1 picks up an image of the neck part of the 
person as the target object, to obtain a texture corresponding to the three-dimensional 
shape of the face part. 

In general, the three-dimensional shape measurement device of the image 
pickup device 1 cannot obtain three-dimensional shape data with respect to a part of 
dark color which absorbs light from an object, e.g., a part such as a head part having 
black hair or metal which reflects light. Therefore, the image pickup device 1 outputs 
three-dimensional shape data of the part such as the face or the like, which will be 
hereinafter called a face part, i.e., the other part of the head part than the hair, and 
texture image data associated with the three-dimensional shape data of the face part. 
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A number of three-dimensional shape measurement devices and two- 
dimensional image input devices may be used for constructing the image pickup 
device 1, in order to measure the neck part of the person as an object (which will be 
described hereinafter as a customer) from many directions. 
5 The image processing device 2 is constructed by: a three-dimensional head part 

reconstruction processing section 3 for synthesizing three-dimensional shape data read 
from a head-part database 6 with three-dimensional shape data inputted from the 
image pickup device 1 , to construct a three-dimensional model of the neck part; a hair 
style mapping processing section 4 for mapping hair-style data read from the hair-style 
1 0 database 5 over the three-dimensional model of the neck part; a rendering section 5 for 
rendering the three-dimensional model of the neck part on which the hair-style data 
3 has been mapped, thereby to generate hair-style-matched image data; a head-part 

database 6 in which three-dimensional shape data of a number of head parts are 
previously recorded; a hair-style database 7 in which a number of hair-style data items 

5 

Q 15 are previously recorded; and a controller 8 which is operated by a user to instruct 

m 

selection of three-dimensional shape data of a head part stored in the head-part 

U1 database 6, a hair-style data item stored in the hair-style database 7, or the like. 

Q 

pj The hair-style data stored in the hair-style database 7 is edited and generated 

with use of a three-dimensional graphic modeler or the like. Therefore, even a hair 

20 style which is actually difficult to make up can be generated and stored. The data 
structure of the hair-style database 7 will be described later with reference to FIG. 1 1 . 
The display device 9 is constructed by a LCD (Liquid Crystal Display) or CRT 
(Cathode Ray Tube) and the like, and displays a hair-style-matched image inputted 
from the image processing device 2. 

25 Next, the operation of the hair synthesis system will be explained with 

reference to the flowchart of FIG. 3 and FIG. 4. In the step SI, the image pickup 
device 1 picks up an image of the a neck part of a customer as a target object or laser- 
scans the neck part, to measure its three-dimensional shape, and outputs three- 
dimensional shape data and texture image data of the neck part of the customer to the 

30 image processing device 2. 

The measurement method of measuring the three-dimensional shape may be 
any method; e.g., a method of using a stereo camera, a method of using a range finder, 
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or the like. As described previously, however, the three-dimensional shape 
measurement device of the image pickup device 1 cannot obtain three-dimensional 
shape data of the hair part. Therefore, three-dimensional shape data and texture image 
data (shown as measurement data in which a texture image is applied to three- 
dimensional shape data of the face part in FIG. 4) of the face part of the customer is 
outputted by the processing in the step SI. If a number of three-dimensional shape 
measurement devices and two-dimensional image input devices are provided in the 
image pickup device 1 , a number of sets each including three-dimensional shape data 
of the face part and texture image data are outputted. 

In the step S2, the three-dimensional head part reconstruction processing 
section 2 of the image processing device 2 constructs a three-dimensional shape model 
of a head part of a customer by using measurement data of the face part inputted from 
the image pickup device 1, i.e., the three-dimensional shape data and texture image 
data. 

Specifically, as shown in FIG. 4, head-part data selected and read from the 
head-part database 6 is combined with the measurement data of the face part of the 
customer, to reconstruct a perfect three-dimensional model of the head part of the 
customer. The head-part database 6 is constructed by three-dimensional models of a 
number of head parts having arbitrary shapes. The head-part data is subjected to 
scaling and positional aligning, based on the size and direction in the measurement 
data of the face part of the customer, and is then combined with the measurement data 
of the customer. 

The combination of the head-part data piece with the measurement data piece 
of the customer may generate a new polygon between open edges in both data pieces, 
or an open edge in one data piece may be changed to fit an open edge of the other data 
piece. 

The three-dimensional shape of the head part of the customer thus constructed 
is expressed, for example, as a triangle-polygon model. In the three-dimensional 
model of the head part of the customer thus constructed, hair is removed from the real 
image of the customer as shown in FIG. 4. 

In the step S3, operation of selecting hair-style data stored in the hair-style 
database 7 by a user (the customer or the like) is received by the controller 8. The 
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selected hair-style data is read out from the hair-style database 7, and is supplied to the 
hair style mapping processing section 4. 

In this hair style selection, icons, thumbnails, or the like indicating hair styles 
are displayed on the display device 9 to invite the user to make a selection. The 
5 selection may be done by moving a cursor or the like on the screen of the display 
device 9 or by inputting a number or the like designating a hair style. 

In the step S4, the hair style mapping processing section 4 of the image 
processing device maps the supplied hair-style data on the three-dimensional mode of 
the head part of the customer supplied from the three-dimensional head part 
10 reconstruction processing section 3, and outputs the result to the rendering part 5. In 
C3 the processing of mapping the hair-style data, hair data corresponding to each one of 

: s 

02 the hairs, which is included in the hair-style data, is mapped on the surface of the 

f * three-dimensional model of the head part, in accordance with the shape of the three- 

tlJ dimensional model of the head part, by the same manner as taken for texture mapping. 

15 Details of the mapping processing will be described later with reference to FIGS. 5 and 
6. 

In the step S5, the rendering section 5 renders the three-dimensional model of 
the head part of the customer, on which the hair-style data has been mapped, to 
generate hair-style-matched image data as shown in FIG. 4, and outputs it to the 
20 display device 9. The method of rendering adopts texture mapping processing for a 
polygon model with respect to the face part of the three-dimensional model, and also 
rendering processing for a curvature model with respect to the hair part. The rendering 
processing for a curvature model is disclosed in details in, for example, "Simulation 
based on a motion model of a string-like object and CG" by Kobayashi, Morishima, 
25 and Harashima, Shingakugihou, PRU90-127, pp. 15-20, 1991. 

The display device 9 displays an image corresponding to the inputted hair- 
style-matched image data. The controller 8 invites a user (a customer or the like) to 
input whether or not the user is satisfied with the hair-style-matched image displayed 
on the display device 9. 
30 At this time, the user can operate the controller 8 to watch the hair-style- 

matched image while changing the viewpoint position and the illumination position. If 
the viewpoint position and the illumination position are changed, the rendering section 
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5 executes again the rendering processing on the three-dimensional model of the head 
part and the hair-style data, so as to correspond to the changed condition (the 
viewpoint position and the illumination position) and outputs the result to the display 
device 9. 

5 In the step S5, if an input indicating that the user is not satisfied is received, the 

processing returns to the step S3, and the later processing is repeated. In the step S5, if 
it is determined that an input indicating the user's satisfaction is received from the user, 
this processing is terminated. 
u Next, an explanation will be made of the hair-style mapping processing based 

H 10 on the hair-style mapping processing section 4 in the step S4. Beforehand, a structural 
m example of the hair-style mapping processing section 4 will be explained with 

%t reference to FIG. 5. 

UJ The hair-style data read out section 21 of the hair-style mapping processing 

yj 

3 section 4 reads hair-style data selected with use of the controller 8, from the hair-style 

== 

" 15 database 7, and outputs it to the mapping position calculation section 22. The mapping 

y i 

M position calculation section 22 calculates the position on the three-dimensional model 

ill 

?j where hair data (corresponding to a hair) forming part of the hair-style data should be 

^ mapped. A hair data interpolation processing section 23 obtains hair data to be 

mapped, by interpolation from the peripheral hair data around the hair-style data. The 
20 coordinate conversion section 24 maps the interpolated hair data by coordinate 
conversion from the object coordinate system of the hair data to the object coordinate 
system of the three-dimensional model of the head part. 

Next, details of the hair style mapping processing of the hair style mapping 
processing section 4 will be explained with reference to the flowchart shown in FIG. 6. 
25 The mapping position calculation section 22 of the hair style mapping processing 
section 4 has already been supplied with hair-style data from the hair-style data read 
out section 21. 

The hair-style data supplied to the mapping position calculation section 22 is 
obtained by two-dimensionally arranging hair data approximated to an object having a 
30 three-dimensional shape, as shown in FIG. 7. 

In the example of FIG. 7A, every hair is approximated to a special curve such 
as a three-dimensional B-spline curve or the like. In the example of FIG. 7B, every 
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hair is approximated to a polygon model. The addresses in the two-dimensional array 
correspond to mapping coordinate values supplied to the three-dimensional model of 
the head part on which the hair style is mapped, like the texture mapping processing. 
For example, if a mapping coordinate (u, v) = (0.1, 0.1) is given to a vertex of a 
predetermined polygon of the three-dimensional model of the head part, the address on 
the hair-style data array of the hair data which is mapped on the vertex is 0.1u max , 
O.lvmax. Where, the u max is the number of data pieces in the u-direction on the hair- 
style data array, and Vmax is the number of data pieces in the v-direction on the hair- 
style data array. 

The hair- style data is defined by arranging the hair data having an arbitrary 
three-dimensional shape into a two-dimensional array. Therefore, any arbitrary hair 
style can be expressed, and it is possible to generate hair-style data of a hair style 
which is difficult to cut. A method of generating hair-style data will be described later 
with reference to FIG. 16. 

At first, in the step SI 1, the mapping position calculation section 22 initializes 
the count value v in the longitudinal direction of the hair-style data array to 0.0. In the 
step SI 2, the count value u in the lateral direction of the hair-style data array is 
initialized to 0.0. 

In the step SI 3, the mapping position calculation section 22 calculates the 
position on the three-dimensional model of the head part where hair data expressed by 
the address (u, v) of the hair-style data array is mapped. 

FIG. 8 shows an arbitrary triangle polygon on the three-dimensional model of 
the head part. The vertexes A, B, and C of the triangle polygon respectively have 
attribute vectors A at tr> B att r, and C attr which express attributes of the three-dimensional 
model of the head part. 

The attribute is constructed by an object coordinate value of the three- 
dimensional model of the head part, local coordinate axes at a vertex, and a mapping 
coordinate value of hair. Where vectors which form two arbitrary edges of the triangle 
polygon ABC are vectors VI and V2, the vectors VI and V2 can be expressed by the 
next expression (1) with use of the attribute vectors of respective vertexes. 

VI — Battr _ Aattr 

V2 = C aMr - A attr ...(1) 
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Considering that the vectors VI and V2 are the coordinate axes of the triangle 
polygon ABC, an arbitrary point Ph on the triangle polygon ABC can be expressed by 
the next expression (2). 

Ph = A attr + aVl + bV2... (2) 

At this time, as shown in FIG. 8, coefficients a and b are defined in a manner 
that lengths between an arbitrary point Ph and points at each of which a line parallel to 
one axis and extended from the arbitrary point crosses the other axis are normalized by 
the lengths of axial vectors, respectively. 

Where the mapping coordinate values of the vertexes A, B, and C are 
respectively (u a , v a ), (u b , v b ), and (lie, v c ), attention is paid to the point P h expressed by 
the expression (2), in relation to the elements of the mapping coordinate values. The 
next expression (3) then exists. 

U h = U a + a (U b + U a ) + b (U c - U a ) 

V h = V a + a (V b - V a ) + b (V c - V a ) . . . (3) 

The expression (3) exists with respect to an arbitrary point in the (U,V) space. 

If the counter value u = u p and the counter value v = v p , the condition for 
satisfying that the point Ph on the three-dimensional model of the head part where hair 
data should be mapped exists inside the triangle polygon is expressed by the next 
expression (4). 

a + b< 1 ... (4) 

That is, in the arbitrary triangle polygon of the three-dimensional model of the 
head part, if the coefficients a and b obtained by substituting u h = u p and v h = v p in the 
expression (3) satisfy the expression (4), the point P h exists inside the triangle polygon. 
Therefore, the hair data expressed by (u p , v p ) on the hair-style data array is mapped on 
the triangle polygon. Accordingly, the mapping position of the hair data on the three- 
dimensional model of the hair data is calculated by using the expression (2) in the 
mapped triangle polygon. 

Returning to FIG. 6, in the step SI 4, the mapping position calculation section 
22 determines whether or not the triangle polygon of the three-dimensional model of 
the head part to be mapped is a hair generation region. This determination is carried 
out, based on a flag (positive or negative) which indicates whether or not hair data 
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added to each triangle polygon of the three-dimensional model of the head part should 
be mapped. 

If the mapping flag of the triangle polygon including the mapping position 
obtained by the processing in the step SI 3 is positive, the triangle polygon is 
determined to be a hair generation region, and the processing goes to the step SI 5. 

In the step SI 5, the hair data interpolation processing section 23 interpolates 
hair data, which is expressed as (u p , v p ) and should be mapped, with use of hair data 
stored in the hair-style data array. The interpolation method may use an average value 
of four close points or may use a summation average value of four close points. 
Alternatively, the closest point may be directly used without interpolation. 

FIG. 9 shows a method of using a summation average value from four close 
points. Supposing now that the counter value u = u p and the counter value v = v p , the 
hair-style data array address of the hair data to be mapped is P a = (m p , n p ) = (u p u^, v p 
Vmax). The hair data at the point P a is Hp, and the close 4 points are PI to P4 as shown 
in FIG. 9. The hair data pieces at the point PI to P4 are respectively HI to H4. 

In the summation average method from 4 close points, the hair data Hp to be 
mapped is interpolated from the data of the 4 close points with use of the next 
expression (5). 

Hp = dm (dn H2 + (1-dn) HI) + (1-dm) (dn H4 + (1-dn) H3)) ... (5) 

If the hair data is a special curve such as a three-dimensional B-spline curve or 
the like as shown in FIG. 7A, the interpolation processing of the hair data is obtained 
by interpolation of data of each control point. Meanwhile, if the hair data is a polygon 
model as shown in FIG. 7B, the interpolation processing of the hair data is obtained by 
interpolation between respective polygon vertex data pieces. In the expression (4), dm 
is the decimal part of m p , and dn is the decimal part of n p . 

Returning to FIG. 6, in the step SI 6, the coordinate conversion section 24 
coordinate converts hair data subjected to interpolation processing, from the object 
coordinate system of the hair data into the object coordinate system of the three- 
dimensional model of the head part. 

The concept of the coordinate conversion will be explained with reference to 
FIGS. 10A and 10B. Suppose that an arbitrary hair data piece on the hair-style data 
array is H. The hair data H is expressed as (I, J, K) according to the object coordinate 



15 



system of the hair data. The object coordinate system (I, J, K) of the hair data may be 
arbitrarily given in case of a three-dimensional coordinate system in which the start 
point of the hair data H is an origin. For example, this is an orthogonal coordinate 
system in which the lateral direction of the hair-style data array is I, the longitudinal 
direction thereof is J, and a direction perpendicular to these directions is K. Thus, the 
start point of the hair data H is expressed as (I, J, K) = (0, 0, 0), and the other control 
points are expressed by relative positions from the start point of the hair data. 

Meanwhile, the three-dimensional model of the head part is expressed as an 
object coordinate system (X, Y, Z). The object coordinate system of the three- 
dimensional model of the head part is generally a three-dimensional orthogonal 
coordinate system, e.g., an orthogonal coordinate system in which the direction toward 
the right ear from the gravity center position of the head part as an origin is X, the 
direction toward the top of the head is Y, and the direction toward the nose is Z. 

The coordinate conversion section 24 converts the hair data H expressed by the 
object coordinate system (I, J, K) of the hair data into a local coordinate system (T, S, 
N) at the mapping position Ph of the three-dimensional model of the head part obtained 
in the step SI 3. The local coordinate system (T, S, N) at this point P h can be arbitrarily 
determined in case of the three-dimensional coordinate system using the point Ph as an 
origin. For example, it is an orthogonal coordinate system in which N is the vector in 
the normal line direction at the point Ph, T is the vector in the tangent direction, which 
is obtained as a vector product of the vector in the radial direction toward the point Ph 
from the center axis (Y-axis in FIG. 10) in the longitudinal direction of the three- 
dimensional model of the head part and the vector N in the normal line direction, and S 
is the vector perpendicular to the normal line direction N and the tangent direction T. 

Suppose that the local coordinate system at each vertex of the three- 
dimensional model of the head part is previously given in the head-part database. 
Therefore, like the calculation of the mapping position, the local coordinate system (T, 
S, N) at the point P h is calculated with use of the expression (2) in the triangle polygon 
to be mapped. Supposing that data at the object coordinate system (I, J, K) of an 
arbitrary control point C\ of the hair data H is (i, j, k), the control point C\ is expressed 
by the next expression (6) in the local coordinate system (T, S, N) at the point Ph. 

Cj = iT+j S+kN ...(6) 
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Where, the unit vectors of the local coordinate system (T, S, N) respectively 
have values of the object coordinate system (X, Y, Z) of the three-dimensional model 
of the head part. 

Therefore, the control point C\ expressed by the expression (6) is expressed by 
the object coordinate system (X, Y, Z) of the three-dimensional model of the head part. 

Returning to FIG. 6, in the step SI 7, the mapping position calculation section 
22 determines whether or not the counter value u exceeds 1 .0. If the counter value u is 
not determined to exceed 1.0, the processing goes to the step SI 8. In the step SI 8, the 
mapping position calculation section 22 increase the counter value u by an increment 
value du. The increment value du indicates how long pitch hair data stored in the hair- 
style data array is mapped in the lateral direction. 

Thereafter, the processing returns to the step SI 3, and the later processing is 
repeated. If it is determined in the step SI 7 that the counter value u exceeds 1.0, 
mapping processing in the lateral direction has reached the end of one line of the hair- 
style data array, so the processing goes to the step SI 9. 

In the step SI 9, the mapping position calculation section 22 determines 
whether or not the counter value v exceeds 1 .0. If the counter value v is determined as 
not exceeding 1.0, the processing goes to the step S20. In the step S20, the mapping 
position calculation section 22 increases the counter value v by an increment value dv. 
The increment value dv indicates how long pitch hair data stored in the hair-style data 
array is mapped in the longitudinal direction. 

Thereafter, the processing returns to the step S12 and the later processing is 
repeated. In the step SI 9, if the counter value v is determined as exceeding 1.0, 
mapping processing is completed with respect to all lines. The hair mapping 
processing is terminated, and the processing goes to the step S5 shown in FIG. 3. 

In the step SI 4, if the mapping flag of the triangle polygon including the 
mapping position obtained by the processing in the step SI 3 is negative, the triangle 
polygon is not determined to be a hair generation region, and the processing of the 
steps SI 5 and S16 is skipped. 

By thus specifying a hair generation region with use of a mapping flag, 
mapping can be effected only on an arbitrary region of the three-dimensional model of 
the head part, without mapping all the hair data pieces that construct the hair-style data 
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read out from the hair- style database 7. Therefore, a hair- style such as a so-called 
Machican cut, a star cut, or the like can be easily generated. 

Also, according to this hair-style mapping processing, each of the hair-style 
data pieces stored in the hair-style database 7 is expressed by the object coordinate 
5 system of hair and can be mapped on a three-dimensional model of a head part having 
any arbitrary shape, without depending on the shape of the human head part. 

Next, the data structure of the hair-style database 7 will be explained with 
reference to FIG. 11. The hair-style database 7 is constructed by a number of hair- 
L4 style data pieces. Each hair-style data piece is of a two-dimensional array as described 

U 10 above, and at the top, the number of data pieces u max in the lateral direction and the 
jll number of data pieces v^* in the longitudinal direction. Following them, hair data 



y ! 



Hi 



pieces for (u max x Vmax) are described. 



yj In some cases, group information (described later) is described at the top of the 

ii j 

~~ hair data piece. Subsequently, a number of control point data pieces are described, in 

u 15 the case where hair data is approximated to a three-dimensional parametric curve or 

Ul 

the like, the control point data pieces are expressed by the hair object coordinate 
system (I, J, K). 

The hair data may be three-dimensional polygon data. In this case, a number 
of polygon vertex data pieces are described as hair data. 
20 The group information described in the hair data can be used for hair data 

interpolation processing in the step S15 of the hair-style mapping processing (FIG. 6) 
described above. 

With reference to FIG. 12, explanation will be made of the hair data 
interpolation processing where group information is supplied.. Where the counter 
25 value u = u p and the counter value v = v p are given, the address on the hair-style data 
array of hair data to be mapped is P a = (m p , n p ) = (u p u,^, v p Vmax). 

Suppose that the hair data at the point P a is Hp, four close points are points PI 
to P4 as shown in FIG. 12, and hair data pieces at the points PI to P4 are respectively 
HI to H4. Further, 1 is described as group information in the hair data piece HI, and 2 
30 is described as group information in each of the hair data pieces H2 to H4. That is, in 
this example, only the hair data piece HI belongs to a group 1, and the other hair data 
pieces H2 to H4 belong to a group 2. If group information is given, only the hair data 
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pieces that belong to one same group are used to perform interpolation, in the hair data 
interpolation processing. 

More specifically, among the four close points of the point P a which is mapped 
at first, the closest point if obtained and the group information thereof is obtained. In 
case of FIG. 12, the closest point is the point P3. Next, among the four close points PI 
to P4, the number of close points that belong to the same group as that of the closest 
point is obtained. 

If no other point among the four close points PI to P4 than the close point 
belongs to the same group as that of the closest point, the hair data piece of the closest 
point is directly used also for the point P a . 

If two close points including the closest point among the four close points PI to 
P4 belong to the same group as that of the closest point, a linear interpolation value 
between the two points is used for interpolation. 

If three close points including the closest point among the four close points PI 
to P4 belong to the same group as that of the closest point, i.e., in the case shown in 
FIG. 12, an interpolation value is calculated by the next expression (7). 

Hp = (l-r)H32 + rH34 

H32 = dn H2 + (l-dn) H3 

H34 = dn H4 + (l-dn) H3 

r = (m p - m32)/(m34 - m32) ... (7) 

In this expression, dn is the decimal part of n p , m32 is the m-address at the 
point P32, and m34 is the m-address at the point P34. 

By thus describing group information in hair data, for example, it is possible to 
express even a hair style having parting of hair where interpolation will cause a 
problem. That is, if grouping at the boundary of parting is performed, hair data pieces 
in both sides of the boundary can be used to prevent generation of unnatural hair, in 
case where four close points spread over the boundary. 

Next, with reference to FIGS. 12 and 13, explanation will be made of an 
applicative example of the coordinate conversion in the processing in the step S 1 6 in 
the hair-style mapping processing (FIG. 6) described above. 

In the coordinate conversion processing in the step SI 6, unevenness in a hair 
style can be expressed with use of random numbers. FIG. 13A shows a state where 
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unevenness in a hair style is not expressed while FIG. 13B shows an example of a state 
in which unevenness in a hair style is expressed by rotating the local coordinate system 
with use of a random number in the coordinate conversion. 

Specifically, the local coordinate system (T, S, N) of the head part is rotated 
about an axis in correspondence with a generated random number where the axis is N. 
Thereafter, the result is converted into the object coordinate system (X, Y, Z) of the 
head part model. By this processing, the directions in which respective hairs grow can 
be changed delicately, without changing the positions where hair data should be 
mapped. 

FIG. 14A shows a state in which unevenness in a hair style is not expressed, 
while FIG. 14B shows an example in which unevenness in a hair style is expressed by 
changing mapping positions by random numbers in the coordinate conversion. 

Specifically, the position of P h as the origin of the local coordinate system (T, 
S, N) is shifted to P n in correspondence with a generated random number, and is 
thereafter converted into the object coordinate system (X, Y, Z) of the head part 
model. By this processing, the position where the hair grows can be changed 
delicately without changing the hair style. 

The technique of rotating the axis of the coordinate system to be converted use 
the random number and/or moving its origin to express natural unevenness in a 
synthesized image can be applied to other cases than the synthesis of hair images. 

Meanwhile, in this hair synthesis system, hair-style data can be mapped after 
changing the three-dimensional model of the head part into an arbitrary shape by the 
three-dimensional head part reconstruction processing section 3. FIG. 15A shows a 
state in which the three-dimensional model of a head part is not changed, while FIG. 
15B shows a state in which the three-dimensional model of the head part is magnified 
1.5 times in the Y-axis direction. 

Magnification/minification processing for a three-dimensional model of a head 
part is achieved by affine conversion processing. At this time, similar affine 
conversion processing is performed also on the local coordinate system (T, S, N) 
assigned to an arbitrary polygon vertex of the three-dimensional model of a head part. 

As shown in FIG. 15B, in the case of magnifying the three-dimensional model 
of the head part for 1.5 times in the Y-axis direction, vectors of T, S, and N are 
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multiplied by a conversion matrix for 1.5 times magnification in the Y-axis direction, 
to obtain a local coordinate system (T, S\ N f ) expanded in the longitudinal direction. 
By mapping hair data H on the local coordinate system (T 1 , S', N') thus deformed, the 
hair style is magnified 1.5 times in the Y-axis direction, into a shape matched with the 
head part. By this processing, same hair-style data can be mapped on the three- 
dimensional model of a head part, which is deformed into an arbitrary shape. Next, 
generation of hair-style data will be explained with reference to FIG. 16. FIG. 16 
shows a columnar coordinate system which surrounds a head part. In the columnar 
coordinate system, the axis penetrating the center of a column is taken as the Y-axis, 
the distance from the Y-axis to a plane perpendicular to the Y-axis is r, and the angle 
from a reference direction on this plane is 0. 

An arbitrary point Ph on the surface of the head part can be expressed as Ph = 
(y p , r p , 0 P ) with use of the columnar coordinate system. Further, if the line defined by 
Y = y p and 0 = 0 P of the columnar coordinate system always has one contact point with 
the surface of the head part, the arbitrary point P h on the surface of the head part can be 
expressed by (y p , 0 P ). 

That is, if the shape of the surface of the head part is always convex toward the 
circumferential direction, the above condition (the line defined by Y= y p and 0 = 0 P of 
the columnar coordinate system always has one contact point with the surface of the 
head part) is satisfied. 

Considering the scalp part excluding the face, the shape of the surface of the 
head part is generally considered to be always convex toward the circumferential 
direction. Therefore, the arbitrary point Ph on the scalp part of the head-part model 
can be expressed by Ph = (y p , 0 P ). 

Creation of hair-style data can be relatively easily carried out by using a system 
disclosed in "Development of a hair design system using tuft models" by Kishi, 
Saegusa, and Morishima, in Shingakugihou, MVE97-102, pp. 67-74, 1998. In the 
system, hair-style data is prepared by editing hair data generated on a given head-part 
model. Hair data outputted from this system is expressed by the object coordinate 
system of the given head-part model. 

The hair database 7 in the present invention records hair-style data which is 
stored in a two-dimensional array by mapping the hair data on a space expressed by (y, 
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0) of a columnar coordinate system. For example, the following expression (8) is used 
for conversion from the (y, 8) space into the (u, v) space of the two-dimensional array 
as shown in FIG. 7. 
u = 6p / 2n 

V = (y p -ymin) / (ymax-ymin) - (8) 

In this expression, y m j n is the smallest value of the Y-axis of the columnar 
coordinate system of the surface of the head part, and ymax is the largest value of the Y- 
axis of the columnar coordinate system of the surface of the head part. 

The conversion from the (y, 0) space into the (u, v) space may be of non-linear. 
For example, sampling near the top part of the head can be prevented from being 
coarse, as long as the conversion is arranged such that the increment in the v-direction 
becomes smaller as it comes closer to the head top. 

As described above, hair-style data consisting of hair data converted into the 
(u, v) space is recorded in the hair-style database 7. 

The present invention is also applicable to the case of expressing a string-like 
object other than hair in CG. 

The series of processing in the image processing apparatus 2 as described 
above can be executed by both hardware and software. In case of executing the series 
of processing by software, the software can be installed from a recording medium into 
a computer in which the program constituting the software is incorporated in dedicated 
hardware or a general-purpose computer which can execute various functions 
computer by installing various programs. 

FIG. 17 shows a structural example of a personal computer which embodies the 
image processing apparatus 2. This personal computer includes a CPU (Central 
Processing Unit) 41. The CPU 41 is connected with an input/output interface 45 
through a bus 44. The input/output interface 45 is connected with: an input section 46 
constructed by an input device such as a keyboard, a mouse, or the like; an image 
output section 47 for outputting a hair-style-matched image to the display device 9; a 
storage section 48 constructed by a hard-disc drive for storing a program, various data, 
and the like; a communication section 49 for communicating data through Internet or 
the like; and a drive 50 for reading/writing data with respect to a recording medium 
such as a magnetic disc 51, a semiconductor memory 54, or the like. The bus 44 is 
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connected with a ROM (Read Only Memory) 42 and a RAM (Random Access 
Memory) 43. 

An image processing program which lets the personal computer execute 
operation as the image processing apparatus 2 is supplied in the form that it is stored in 
the magnetic disc 51 or semiconductor memory 54, to the personal computer. It is 
read by the drive 50 and installed in the hard disc drive included in the storage section 
48. The image processing program installed in the storage section 48 is loaded from 
the storage section 48 to the RAM 43 and executed, in accordance with instructions 
from the CPU 41 corresponding to commands from the user. 

In the present invention, the step of describing the program recorded in the 
recording medium includes not only processing which is carried out based on time in 
accordance with a described order but also processing which is executed in parallel or 
individually although it is not always performed based on time. 

Also, in the present invention, the term of "system" includes the entire 
apparatus constructed of a number of apparatuses and/or devices. 
Industrial Applicability 

As described above, according to the present invention, the hair-style is stored, 
stored hair-style data is read out, and hair data corresponding to a hair and included in 
the hair-style data is mapped at a predetermined position of a three-dimensional shape 
image of a head part. Therefore, a hair style consisting of hairs which are expressed 
by CG can be changed so as to correspond to a number of different head shapes. Also, 
according to the present invention, stored hair-style data is read out, and hair data 
corresponding to a hair and included in the read out hair-style data is mapped at a 
predetermined position on a three-dimensional shape image of a head part. Based on 
the mapping results, a hair-style-matched image is generated. Therefore, a hair style 
consisting of hairs which are expressed by CG can be changed so as to correspond to a 
number of different head shapes. 

Although the present invention has been described with reference to specific 
embodiments, those of skill in the art will recognize that changes may be made thereto 
without departing from the spirit and scope of the invention as set forth in the hereafter 
appended claims. 
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